Information processing apparatus, information processing method, and computer program

ABSTRACT

Provided is an information processing apparatus including an event processing unit for performing in a background a process corresponding to an operation input that is estimated based on a state of an operating tool in a first state, and for performing, when the operating tool in the first state enters a second state, a feedback process for the operation input using the process performed in the background.

BACKGROUND

The present disclosure relates to an information processing apparatus,an information processing method, and a computer program for causing aprocess according to an operation input to be performed.

Due to being able to realize a user interface (hereinafter, alsoreferred to as a “UI”) that was intuitive and easy to use, touch panelshave been used for ticket-vending machines for public transportation,ATMs at banks, and the like. These days, touch panels have come to beable to detect actions of users, and to realize operation of appliancesthat were not attained by button operations in the past. Thus, thesedays, touch panels are being used more often in portable appliances suchas mobile phones and game machines.

Furthermore, as indicated in JP 2008-117371A, a touch panel is proposedthat not only detects contact of an operating tool, such as a finger ora touch pen, on a detection surface, but also detects proximity of theoperating tool on the detection surface. Various operation inputs arethereby made realizable.

SUMMARY

However, when using a touch panel as an input unit with which anoperation input is performed, preprocessing for performing a processcorresponding to an operation input is started after the operation inputis detected. Accordingly, one has to wait, after performing an operationinput, for a processing time for the preprocessing until a processcorresponding to the operation input is performed, and a delay may occurin GUI feedback to the user.

For example, as shown in FIG. 13, a case is considered where onecompressed file is selected from a list 11 of compressed files, such aszip files, displayed on a display 10 and a process is performed todecompress the compressed file. When a finger contacts a decompressionbutton 13 for starting decompression on a confirmation screen 12 usedfor confirming whether or not to decompress the compressed file that auser has selected from the list 11, preprocessing for decompressing theselected compressed file is started. Then, when the compressed file isdecompressed, display processing of the decompressed file is started,and the contents of the file are displayed on the screen 10. At thistime, since the decompression processing of the compressed file isperformed after the operation input is detected, there is a slight delayuntil feedback is provided for the operation input.

Such a delay occurs, besides in decompression of compressed files, infile decompression in Samba, music/video decompression, a page-turningprocess of a pdf file, a decoding process of a piano softwaresynthesizer, and the like. For example, with a decoding process of apiano software synthesizer, it is important that there is audio outputfeedback at the moment of operation input, and operability may bereduced by a delay by the preprocessing.

Furthermore, as shown in FIG. 14, for example, a case is consideredwhere, in a computer strategy game such as Japanese chess, an operationinput of a computer is processed using Artificial Intelligence. In thiscase, since a large amount of calculation processing has to be performedusing AI processing to determine an operation input of a computer inresponse to an operation input of a user, there is a period in which thecomputer remains in a thinking state. Thus, a slight delay may occuruntil the thinking by the computer is complete, that is, the AIprocessing is complete, and the next operation input of the user isenabled.

A delay in processing is prone to occur also in a case feedback for anoperation input of a user is provided over a network. For example, asshown in FIGS. 15 and 16, cases are considered where map information isacquired from a server that is connected to the screen 10 of a displaydevice over a network. As shown in FIG. 15, in the case of enlargingdisplay of map information displayed on the screen 10 by a pinch-outaction of placing two fingers on the screen and widening the gap betweenthe fingers, a process of acquiring enlarged map information from theserver is started after the pinch-out action is detected. Thus, a slightdelay may occur from detection of an operation input to a read processof the enlarged map information and to display of an image on the screen10.

Likewise, as shown in FIG. 16, in the case of reducing display of mapinformation displayed on the screen 10 by a pinch-in action of placingtwo fingers on the screen and narrowing the gap between the fingers, aprocess of acquiring reduced map information from the server is startedafter the pinch-in action is detected. Thus, a slight delay may occurfrom detection of an operation input to a read process of the reducedmap information and to display of an image of the screen 10.

As such, with an application according to which there is a certainprocessing time until a user receives feedback for an operation input,such as a tap or a multi-touch gesture, after performing the operationinput, it is desired that such a delay is prevented so as not to reduceoperability.

According to the present disclosure, there is provided an informationprocessing apparatus which includes an event processing unit forperforming in a background a process corresponding to an operation inputthat is estimated based on a state of an operating tool in a firststate, and for performing, when the operating tool in the first stateenters a second state, a feedback process for the operation input usingthe process performed in the background.

Also, according to the present disclosure, there is provided aninformation processing method which includes performing in a backgrounda process corresponding to an operation input that is estimated based ona state of an operating tool in a first state, and performing, when theoperating tool in the first state enters a second state, a feedbackprocess for the operation input using the process performed in thebackground.

Furthermore, according to the present disclosure, there is provided acomputer program for causing a computer to function as an informationprocessing apparatus including an event processing unit for performingin a background a process corresponding to an operation input that isestimated based on a state of an operating tool in a first state, andfor performing, when the operating tool in the first state enters asecond state, a feedback process for the operation input using theprocess performed in the background.

According to the present disclosure, by performing a process for anoperation input that is estimated from the state of an operating tool ina first state, a feedback process can be performed without a delay whenthe operating tool enters a second state and the estimated operationinput is performed.

As described above, according to the present disclosure, by estimatingan operation input and performing, in the background, preprocessing fora process that is to occur, a slight delay until GUI feedback isprovided can be prevented.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an explanatory diagram showing a relationship between aprocess performed by an information processing apparatus according to afirst embodiment of the present disclosure and an operation input;

FIG. 2 is a hardware configuration diagram showing an example of ahardware configuration of the information processing apparatus accordingto the embodiment;

FIG. 3 is a functional block diagram of a functional configuration ofthe information processing apparatus according to the embodiment;

FIG. 4 is a flow chart showing a process, of the information processingapparatus according to the embodiment, up to providing feedback for anoperation input;

FIG. 5 is an explanatory diagram describing a process that is estimatedwhen there is one hovering finger;

FIG. 6 is an explanatory diagram describing a process that is estimatedwhen two hovering fingers are adjacent to each other;

FIG. 7 is an explanatory diagram describing a process that is estimatedwhen two hovering fingers are separate from each other;

FIG. 8 is an explanatory diagram describing processing contents up tofeedback where a pinch-out is estimated;

FIG. 9 is an explanatory diagram describing processing contents up tofeedback where a pinch-in is estimated;

FIG. 10 is an explanatory diagram describing feedback in a decompressionprocess of a compressed file according to the embodiment;

FIG. 11 is an explanatory diagram describing feedback in AI processingaccording to the embodiment;

FIG. 12 is an explanatory diagram showing a relationship between aprocess performed by an information processing apparatus according to asecond embodiment of the present disclosure and an operation input;

FIG. 13 is an explanatory diagram describing a delay in feedback in adecompression process of a compressed file;

FIG. 14 is an explanatory diagram describing a delay in feedback in AIprocessing;

FIG. 15 is an explanatory diagram describing a delay in feedback in aprocess over a network, and shows a case where a pinch-out action isperformed; and

FIG. 16 is an explanatory diagram describing a delay in feedback in aprocess over a network, and shows a case where a pinch-in action isperformed.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and configuration are denotedwith the same reference numerals, and repeated explanation of thesestructural elements is omitted.

Additionally, an explanation will be given in the following order.

1. First Embodiment (Detection of Proximity and Contact of OperatingTool on Detection Surface)

1.1. Overview of Process of Information Processing Apparatus

1.2. Hardware Configuration

1.3. Functional Configuration

1.4. Feedback Process for Operation Input

2. Second Embodiment (Detection of Contact and Pressing of OperatingTool on Detection Surface) 1. First Embodiment 1.1. Overview of Processof Information Processing Apparatus

First, an overview of a process of an information processing apparatusaccording to a first embodiment of the present disclosure will be givenwith reference to FIG. 1. Additionally, FIG. 1 is an explanatory diagramshowing a relationship between a process performed by the informationprocessing apparatus according to the present embodiment and anoperation input.

The information processing apparatus according to the present embodimentis a device that is connected with a display device, such as a touchpanel, and that processes information transmitted from the displaydevice. Such an information processing apparatus estimates an operationinput of a user and performs, in the background, preprocessing forproviding feedback for the estimated operation input so as to prevent adelay between reception of an operation input and provision of feedbackfor the operation input. As shown in FIG. 1, in the present embodiment,when a first state, that is, proximity of an operating tool, such as afinger or a touch pen, to a detection surface 200, is detected, theinformation processing apparatus acquires the state of the detectedoperating tool that is in proximity. Then, an operation input that auser is about to perform is estimated based on the state of theoperating tool acquired, and preprocessing is performed, in thebackground, for providing feedback for the operation input which hasbeen estimated.

Then, when a second state is detected, that is, when the operating toolcontacted the display surface of the display device, the informationprocessing apparatus provides GUI feedback based on a processing resultof the preprocessing that was performed in the background. In thismanner, by estimating an operation input of a user and performingpreprocessing in the background in advance, time until the provision ofGUI feedback can be shortened. In the following, a configuration of theinformation processing apparatus according to the present embodiment andits function will be described in detail.

1.2. Hardware Configuration

First, a hardware configuration of an information processing apparatus100 according to the first embodiment of the present disclosure will bedescribed with reference to FIG. 2. Additionally, FIG. 2 is a hardwareconfiguration diagram showing an example of a hardware configuration ofthe information processing apparatus 100 according to the presentembodiment.

As shown in FIG. 2, the information processing apparatus 100 accordingto the present embodiment includes a RAM (Random Access Memory) 101, anon-volatile memory 102, a display device 103, a CPU 104, and aproximity detection touch panel 105.

The RAM 101 temporarily stores programs used in the execution of the CPU104, parameters arbitrarily changed in the execution of the programs, orthe like. These elements are connected to each other by a host busconfigured from a CPU bus or the like. The non-volatile memory 102stores programs, arithmetic parameters or the like to be used by the CPU104. As the non-volatile memory 102, a ROM (Read Only Memory), a flashmemory or the like may be used, for example.

The display device 103 is an example of an output device that outputsinformation. As the display device 103, a liquid crystal display (LCD)device, an organic light emitting diode (OLED) device or the like may beused, for example. In the present embodiment, the display device 103does not necessarily have to be provided in the information processingapparatus 100, and may be provided as an external appliance connected tothe information processing apparatus 100 so as to be able totransmit/receive information.

The CPU 104 functions as an arithmetic processing unit and a controlunit, and controls entire operation within the information processingapparatus 100 according to various programs. Moreover, the CPU 104 mayalso be a microprocessor. The proximity detection touch panel 105 isprovided, being stacked on the display surface of the display device,and detects proximity and contact of an operating tool, such as afinger, by sensing an electrical signal by static electricity, forexample.

1.3. Functional Configuration

Next, a functional configuration of the information processing apparatus100 according to the present embodiment will be described based on FIG.3. Additionally, FIG. 3 is a functional block diagram showing a functionconfiguration of the information processing apparatus 100 according tothe present embodiment. FIG. 3 illustrates functional units necessary toperform a process up to provision of GUI feedback for an operation inputof a user, and the information processing apparatus 100 according to thepresent embodiment may also include other functional units.

As shown in FIG. 3, the information processing apparatus 100 accordingto the present embodiment includes a proximity detection unit 110, anoperation estimation unit 120, an event processing unit 130, anoperation status determination unit 140, an output unit 150, and asetting storage unit 160.

The proximity detection unit 110 is a detection unit for detectingproximity and contact of an operating tool on a detection surface, andcorresponds to the proximity detection touch panel 105 in FIG. 2. Thedetection surface of the proximity detection unit 110 may be provided onthe display surface of a display device or on the surface of the housingof a device, for example. In the case of providing the detection surfaceon the display surface of a display device, a GUI can be provided whichcreates a feeling of as if directly operating, by an operating tool,information that is displayed to a user, and thus intuitive operationinput is enabled.

When an operating tool comes closer to the detection surface than aproximity detection position 210 that is vertically above and separateby a predetermined distance h from the detection surface 200, theproximity detection unit 110 can detect the position of the operatingtool with respect to the detection surface 200. Furthermore, theproximity detection unit 110 can also detect contact of the operatingtool on the detection surface 200. Although it depends on the detectionmethod of the proximity detection unit 110, the proximity state and thecontact state can be detected based on the size of an electrical signalwhen using a detection method of sensing an electrical signal by staticelectricity, for example. A detection result of the proximity detectionunit 110 (for example, an electrical signal by static electricity) isoutput to the operation estimation unit 120 and the operation statusdetermination unit 140 at a predetermined interval, such as every time adetection value is acquired.

The operation estimation unit 120 determines whether or not an operatingtool is in proximity of the detection surface, based on a detectionresult of the proximity detection unit 110. Also, in the case anoperating tool is determined, by the result of determination, to be inproximity of the detection surface, the operation estimation unit 120estimates an operation input which will possibly be performed based onthe state of the operating tool that is in the proximity state. Thecorrespondence relationship between the state of an operating tool andan operation input is stored in advance in the setting storage unit 160,for example, and the operation estimation unit 120 can estimate anoperation input using this correspondence relationship. The operationestimation unit 120 outputs the estimated operation input to the eventprocessing unit 130.

Furthermore, the operation estimation unit 120 determines, based on anoperation status of the operating tool input from the operation statusdetermination unit 140 described later, the possibility of an operationinput which has already been estimated being performed. At this time, ifit is determined that the possibility of an operation input which hasalready been estimated being performed is low, the operation estimationunit 120 outputs an instruction, to the event processing unit 130, forcancelling preprocessing for feedback that is being performed.

The event processing unit 130 performs, in the background, preprocessingnecessary for providing feedback corresponding to the operation inputestimated by the operation estimation unit 120, before the operationinput is actually performed. Feedback for an operation input is storedin advance in the setting storage unit 160, for example, and the eventprocessing unit 130 refers to the setting storage unit 160 andrecognizes corresponding feedback based on the estimated operationinput. Then, a process for providing the feedback is performed in thebackground.

Furthermore, when the estimated operation input is actually performed,the event processing unit 130 provides feedback using the preprocessingwhich has been performed in the background. A slight delay occurringbetween the operation input of a user and provision of feedback canthereby be prevented. The event processing unit 130 outputs to theoutput unit 150 the feedback for the operation input.

The operation status determination unit 140 determines the operationstatus by the operating tool detected by the proximity detection unit110, based on the detection result of the proximity detection unit 110,and outputs the determination result to the operation estimation unit120. As described above, the operation estimation unit 120 determines,based on the operation status by the operating tool input by theoperation status determination unit 140, the possibility of theoperation input which has already been estimated being performed. Likethe operation estimation unit 120, the operation status determinationunit 140 is also capable of estimating an operation input using thecorrespondence relationship between the state of an operating tool andan operation input stored in the setting storage unit 160.

The operation estimation unit 120, the event processing unit 130, andthe operation status determination unit 140 can be realized by the CPU104 in FIG. 2, for example.

The output unit 150 is an interface for outputting feedback from theevent processing unit 130. The output unit 150 may be a display deviceor an audio output device such as a speaker that directly providesfeedback to a user, for example, or may be an informationtransmission/reception unit connected to an external display device orthe audio output device and the information processing apparatus 100.

The setting storage unit 160 is a storage unit storing various types ofinformation necessary for performing a process up to provision of GUIfeedback for an operation input of a user, and corresponds to the RAM101 or the non-volatile memory 102 in FIG. 2, for example. Informationto be stored in the setting storage unit 160 may be the correspondencerelationship between the state of an operating tool and an operationinput, the relationship between an operation input and its feedback, andthe like, for example. These pieces of information may be stored inadvance as the initial setting of the information processing apparatus100, or may be set by a user.

1.4. Feedback Process for Operation Input

As described above, when the first state, that is, proximity of anoperating tool to the display surface of a display device, is detected,the information processing apparatus 100 according to the presentembodiment acquires the state of the detected operating tool which is ata proximity position. Then, an operation input that a user is about toperform is estimated based on the state of the operating tool which hasbeen acquired, and preprocessing for providing feedback to the estimatedoperation input is performed in the background. Then, when the secondstate is detected, that is, when the operating tool contacts the displaysurface of the display device, the information processing apparatusprovides GUI feedback based on a processing result of the preprocessingthat was performed in the background. In this manner, by estimating anoperation input of a user and performing preprocessing in the backgroundin advance, time until the provision of GUI feedback is shortened.

In the following, processing contents, of the information processingapparatus 100 according to the present embodiment, up to provision offeedback to an operation input will be described based on FIGS. 4 to 9.FIG. 4 is a flow chart showing a process, of the information processingapparatus 100 according to the present embodiment, up to providingfeedback for an operation input. FIG. 5 is an explanatory diagramdescribing a process that is estimated when there is one hoveringfinger. FIG. 6 is an explanatory diagram describing a process that isestimated when two hovering fingers are adjacent to each other. FIG. 7is an explanatory diagram describing a process that is estimated whentwo hovering fingers are separate from each other. FIG. 8 is anexplanatory diagram describing processing contents up to feedback wherea pinch-out is estimated. FIG. 9 is an explanatory diagram describingprocessing contents up to feedback where a pinch-in is estimated.

Here, a case where an application related to display of map informationthat can be acquired from a server connected over a network is run willbe described as an example. Moreover, the operating tool is assumed tobe a finger.

First, the operation estimation unit 120 of the information processingapparatus 100 detects, based on a detection result of the proximitydetection unit 110, whether there is a finger, which is an operatingtool, above the detection surface (S100: hover detection). The proximitydetection unit 110 can detect the position of a finger with respect tothe detection surface when the finger comes closer to the detectionsurface than a proximity detection position 210 that is vertically aboveand separate by a predetermined distance h from the detection surface200. Then, when it is determined based on a detection result of theproximity detection unit 110 that there is a finger not in contact withthe detection surface but closer to the detection surface than theproximity detection position 210, that is, when it is determined thatthere is a hovering finger, the operation estimation unit 120 performsthe processing of step S102.

In step S102, the operation estimation unit 120 determines, based on thedetection result of the proximity detection unit 110, the number ofhovering fingers and their states. Whether a finger is in contact withthe detection surface and the position of a detected finger with respectto the detection surface can be recognized based on the result of theproximity detection unit 110. The operation estimation unit 120 canthereby recognize the number of fingers detected and the states of thedetected fingers. The operation estimation unit 120 estimates anoperation input that the user is about to perform, by recognizinghovering finger(s) and the number thereof.

In the present example, as the operation inputs that a user performswhen map information acquired from a server is displayed, there are atap, a pinch-out, a pinch-in, and the like, for example. As shown inFIG. 5, for example, in the case only one point, P1, is detected withina proximity region (a spatial region between the detection surface andthe proximity detection position 210 that is vertically above andseparate by a predetermined distance h from the detection surface), itcan be estimated that there is only one hovering finger. In this case,it can be assumed that an action of tapping map information with onefinger is about to be performed. Also, as shown in FIG. 6, for example,in the case two points, P1 and P2, are detected within the proximityregion and a distance L between the two points P1 and P2 is smaller thana predetermined distance Ls, it can be estimated that two fingers arehovering and that these fingers are adjacent to each other. In thiscase, it can be assumed that an action of pinching out the mapinformation with two fingers is about to be performed.

Furthermore, as shown in FIG. 7, for example, in the case two points, P1and P2, are detected within the proximity region and the distance Lbetween the two points P1 and P2 is the predetermined distance Ls orgreater, it can be estimated that two fingers are hovering and thatthese fingers are separate from each other. In this case, it can beassumed that an action of pinching in the map information with twofingers is about to be performed. Such correspondence relationshipsbetween the states of fingers and the operation inputs are stored in thesetting storage unit 160 in advance, and the operation estimation unit120 can estimate an operation input that a user is about to perform byreferring to the setting storage unit 160 based on the detection resultof the proximity detection unit 110. Additionally, the predetermineddistance Ls may be set based on the distance between fingers at thefirst time of the user performing the pinch-out action or the pinch-inaction in the past, for example.

In the present example, in the case it is recognized in step S102 thatthe number of hovering fingers is one, the operation estimation unit 120estimates that the operation input (Event) that the user is about toperform is a tap (S104). Also, in the case it is recognized in step S102that the number of hovering fingers is two and that they are adjacent toeach other, the operation estimation unit 120 estimates that theoperation input that the user is about to perform is a pinch-out (S106).Furthermore, in the case it is recognized in step S102 that the numberof hovering fingers is two and that they are separate from each other,the operation estimation unit 120 estimates that the operation inputthat the user is about to perform is a pinch-in (S108). The operationinput estimated in step S104, S106, or S108 is output from the operationestimation unit 120 to the event processing unit 130 (S110).

The event processing unit 130 which has been notified by the operationestimation unit 120 of the estimated operation input performs aprefetched-event processing in the background on the object side (S112).In step S112, the event processing unit 130 performs in advanceprocessing of feedback that is to be provided in a case the estimatedoperation input is carried out, before the operation input is actuallyperformed. For example, in the case it is estimated by the operationestimation unit 120 that a tap will be performed, the event processingunit 130 acquires from the setting storage unit 160 contents ofprocessing of feedback that is based on the tap action according to themap information display application. For example, in the case theaddress or area information of a location that is displayed at thetapped position is to be displayed by the tap action, the eventprocessing unit 130 performs in the background a process for acquiringthe address or area information of the location from a server.

Furthermore, for example, in the case it is estimated by the operationestimation unit 120 that a pinch-out will be performed, the eventprocessing unit 130 acquires from the setting storage unit 160 contentsof processing of feedback that is based on the pinch-out actionaccording to the map information display application. For example, inthe case a part displayed at a position contacted by fingers is to beenlarged and displayed by the pinch-out action, the event processingunit 130 estimates that the positions the fingers are in proximity ofare the positions that the fingers will contact, and performs in thebackground a process for acquiring from a server map information, asshown in FIG. 8, showing the part displayed at the position in anenlarged manner.

Moreover, for example, in the case it is estimated by the operationestimation unit 120 that a pinch-in will be performed, the eventprocessing unit 130 acquires from the setting storage unit 160 contentsof processing of feedback that is based on the pinch-in action accordingto the map information display application. For example, in the case apart displayed at a position contacted by fingers is to be reduced anddisplayed by the pinch-in action, the event processing unit 130estimates that the positions the fingers are in proximity of are thepositions that the fingers will contact, and performs in the backgrounda process for acquiring from a server map information, as shown in FIG.9, showing the part displayed at the position in a reduced manner.

In this manner, in step S112, a process that is to be performed when anoperation input is performed is performed in the background based on theestimated operation input. On the other hand, the operation statusdetermination unit 140 monitors, based on a detection result of theproximity detection unit 110, a change in the state of a finger that iscurrently detected (S114). In the present example, the operation statusdetermination unit 140 acquires the position of a finger that is in ahovering state, and determines whether there is a movement of the fingerand whether the finger that is currently detected is at a positioncorresponding to a different object. Whether the process to be performedin the background according to the position of the currently detectedfinger will change or not can thereby be identified.

It is assumed that the operation status determination unit 140 hasdetermined that the finger detected in step S114 is moving in a hoveringstate and is at a position corresponding to a different object. In thiscase, the operation estimation unit 120 outputs to the event processingunit 130 an instruction for cancelling the process that is currentlybeing performed in the background (S116). The event processing unit 130cancels the processing that is being performed in the background, andthen, the information processing apparatus 100 starts again from theprocessing of step S100.

Alternatively, it is assumed that the operation status determinationunit 140 has determined that the finger detected in step S114 is not ina state of moving in a hovering state and of existing at a positioncorresponding to a different object. In this case, the processing in thebackground that was started in step S112 is continued. Then, when theoperation status determination unit 140 determines, based on a detectionresult of the proximity detection unit 110, that the hovering fingercomes in contact, the operation estimation unit 120 instructs the eventprocessing unit 130 to perform processing of the feedback for theestimated operation input.

The event processing unit 130, which has received the instruction, firstdetermines whether the prefetched-event processing in the background iscomplete (S120), and if it is not complete, continues the process in thebackground. Then, when the process in the background is complete, theevent processing unit 130 provides feedback to the operation input byusing the prefetched-event processing which was performed in thebackground (S122). In this manner, by performing a process in thebackground in advance, feedback for an actual operation input can beprovided without a delay.

Additionally, the processing of steps S118 to S122 is performed in thesame manner also when it is determined in step S100 that there is nohover detection. For example, in the case there is no hovering fingerand contact on the detection surface is detected, an operation input atthe time of contact is determined, and a process for providingcorresponding feedback is performed.

When there is no contact detection in step S118, or when feedback for anoperation input is provided in step S122, the operation statusdetermination unit 140 determines whether the hovering finger has movedto a region where detection by the proximity detection unit 110 is notpossible (to the outside of a hover detection area) (S124). In the caseit is determined in step S124 that the hovering finger has not moved tothe outside of the hover detection area, the information processingapparatus 100 ends the process shown in FIG. 4, and performs the processagain from step S100. On the other hand, in the case it is determined instep S124 that the hovering finger has moved to the outside of the hoverdetection area, the operation estimation unit 120 outputs, to the eventprocessing unit 130, an instruction for cancelling the process that iscurrently being performed in the background (S116). The event processingunit 130 cancels the process that is being performed in the background,and then, the information processing apparatus 100 starts again from theprocess of step S100.

In the foregoing, the information processing apparatus 100 according tothe first embodiment of the present disclosure and its feedback processfor an operation input have been described. According to the presentembodiment, based on a detection result of the proximity detection unit110, the state of a finger that is in a proximity state is recognized,and an operation input is estimated. Then, preprocessing for providingfeedback corresponding to the estimated operation input is performed inthe background. Then, when the estimated operation input is performed,feedback is provided using the process that was performed in thebackground. A slight delay between an operation input by a user andprovision of feedback corresponding to the operation input can therebybe prevented.

Although a method of preventing a slight delay occurring when acquiringmap information from a server connected over a network has beendescribed with reference to FIGS. 4 to 9, a delay occurring whendecompressing a compressed file or a delay occurring when performing AIprocessing can also be prevented by the process shown in FIG. 4.

For example, a case is considered, as shown in FIG. 10, of performing aprocess of selecting and decompressing one compressed file in a list 201of compressed files, such as zip files, displayed on a screen 200. Whenproximity of a finger to a decompression button 201 for startingdecompression is detected on a confirmation screen 202 used forconfirming whether or not to decompress a compressed file selected by auser from the list 201, preprocessing for decompressing the selectedcompressed file is started in the background. Then, when the fingeractually touches the decompression button 203, the informationprocessing apparatus 100 starts a display process of a file decompressedin the background, and the contents of the file are displayed on thescreen 200.

In this manner, by performing a decompression process of a compressedfile in the background when a finger comes into proximity of thedecompression button 203, a slight delay between contacting of thefinger on the decompression button 203 and display of contents of a fileon the screen 200 can be prevented.

Furthermore, for example, as shown in FIG. 11, in the case of performingAI processing of an operation input of a computer in a computer strategygame, such as Japanese chess, the information processing apparatus 100estimates that a process for a case of selecting an object correspondingto a position of a finger that is in proximity of the detection surfaceis to be performed. When an operation input is estimated, the eventprocessing unit 130 performs in the background a process that isperformed when the estimated operation input is performed.

Here, the user may, while thinking about the next operation input, movethe finger in a hovering state according to what he/she is thinking.When the object corresponding to the position of the finger is changedby the finger in the hovering state moving, the estimated operationinput also changes. In this case, the operation estimation unit 120cancels the process that is being performed in the background, andperforms in the background a process which will be performed when anewly estimated operation input is performed. In this manner, theoperation status is monitored, and preprocessing for providing feedbackfor an operation input that is highly likely to be performed isperformed in the background. When the operation input is actuallyperformed, the information processing apparatus 100 provides feedbackusing the process performed in the background. The thinking time of thecomputer can thereby be shortened, and a slight delay occurring untilthe next operation input of the user is enabled can be prevented.

Furthermore, the information processing apparatus 100 according to thepresent embodiment decides, based on the state of a finger in theproximity state (the number of fingers, the proximity position, or thelike), the target to be taken into consideration when performing aprocess in the background. Here, the information processing apparatus100 may widen the target range of the process that is to be performed inthe background as the number of hovering fingers increases, for example.Or, the target range of the process that is to be performed in thebackground may be widened according to the range of movement of thehovering finger. The process to be performed in the background canthereby be adjusted, and the possibility of a delay occurring in theprocess can be reduced.

For example, in the example shown in FIG. 11, the target range of aprocess that is to be performed in the background is widened accordingto the range of movement of the finger in a hovering state that is inaccordance with the thinking of the user. By performing in thebackground preprocessing for each of processes that are to be performedwhen objects included in the range of movement of the finger areoperated, feedback can be provided without a delay for any of aplurality of possible operation inputs.

Furthermore, the information processing apparatus 100 may preferentiallyperform, in the background, a process that is to be performed when afinger contacts a position at which a finger is closest to the detectionsurface, according to the status of the finger. For example, it isassumed that proximity of a plurality of fingers has been detected inthe example shown in FIG. 11. In this case, if preprocessing isperformed for all the operation inputs estimated based on the positionsof the respective fingers, heavy load may be placed on the informationprocessing apparatus 100. Thus, in the case proximity of a plurality offingers is detected, a process that is to be performed when a fingercontacts a position at which a finger is closest to the detectionsurface is decided to be the process that is most likely to beperformed, and is preferentially performed in the background. The loadon the information processing apparatus 100 can thereby be reduced.

Furthermore, with the information processing apparatus 100 according tothe present embodiment, the operation estimation unit 120 may estimate aplurality of operation inputs which may be performed by a user, based ona detection result of the proximity detection unit 110. In this case,the operation estimation unit 120 may notify the event processing unit130 of an operation input for which the execution priority is thehighest among the estimated operation inputs. The execution priority isset to specify an operation input that is most likely to be performed.The execution priority can be set based on the number of fingers thatare in proximity of the detection surface, or, if a plurality of fingersare detected, the positional relationship thereof, for example. Or, theexecution priority may be set in advance, highest from the processingcontents desired to be preferentially performed. By setting theexecution priority in this manner, an estimated operation input may beuniquely determined.

Additionally, the centre position of a plurality of fingers detected tobe in proximity, and not the position of a finger that is closest to thedetection surface, may be taken as the position that is most likely tobe contacted by the fingers.

2. Second Embodiment

Next, an overview of a process of an information processing apparatusaccording to a second embodiment of the present disclosure will be givenwith reference to FIG. 12. Additionally, FIG. 12 is an explanatorydiagram showing a relationship between a process performed by aninformation processing apparatus according to the present embodiment andan operation input.

As in the first embodiment, the information processing apparatusaccording to the present embodiment is a device that is connected to adisplay device, such as a touch panel, and that processes informationtransmitted from the display device. To prevent a delay betweenreception of an operation input and provision of feedback for theoperation input, such an information processing apparatus estimates anoperation input of a user and performs, in the background, preprocessingfor providing feedback to the estimated operation input.

As shown in FIG. 12, when a first state, that is, contact of anoperating tool, such as a finger or a touch pen, on a detection surface200, is detected, the information processing apparatus according to thepresent embodiment acquires the state of the detected operating toolthat is at the contact position. Then, based on the state of theoperating tool acquired, an operation input that is about to beperformed by the user is estimated, and preprocessing for providingfeedback for the estimated operation input is performed in thebackground.

Then, when a second state is detected, that is, when the operating toolplaced pressure that is higher than predetermined pressure Pth on thedisplay surface of the display device, the information processingapparatus provides GUI feedback based on the processing result of thepreprocessing that was performed in the background. In this manner, byestimating an operation input of a user and performing preprocessing inthe background in advance, time until the provision of GUI feedback canbe shortened.

The information processing apparatus according to the present embodimentcan be configured in the same way as the information processingapparatus 100 according to the first embodiment. However, theinformation processing apparatus according to the present embodimentuses, instead of the proximity detection touch panel 105 of FIG. 2, apressure-sensitive touch panel that senses a change in the pressure on adisplay surface and detects contact of a finger. The first state, whichis a trigger for performing a process in the background, and the secondstate, which is a trigger for actually starting feedback, can thereby bedistinguished. Other configurations and functions are the same as thoseof the first embodiment.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

For example, in the above embodiments, an information processingapparatus is applied to decompression of a compressed file, AIprocessing, information acquisition over a network, or the like, but thepresent technology is not limited to such examples. For example, byapplying an information processing apparatus according to the aboveembodiments to acquisition process of a music piece from a musicplayback list or a video acquisition process, a slight processing delaycan be prevented in the same manner.

Additionally, configurations as below are also within the technicalscope of the present disclosure.

(1)

An information processing apparatus including:

an event processing unit for performing in a background a processcorresponding to an operation input that is estimated based on a stateof an operating tool in a first state, and for performing, when theoperating tool in the first state enters a second state, a feedbackprocess for the operation input using the process performed in thebackground.

(2)

The information processing apparatus according to (1), including:

a detection unit for detecting a state of the operating tool;

an operation estimation unit for estimating, based on a detection resultof the detection unit, an operation input by the operating tool; and

an operation status determination unit for determining, and notifying tothe operation estimation unit, a change in a state of the operating toolthat is in the first state,

wherein, when the operating tool is determined based on the detectionresult of the detection unit to be in the first state, the operationestimation unit estimates, and notifies to the event processing unit, anoperation input by the operating tool, and

wherein, when the operation status determination unit determines basedon the detection result of the detection unit that the operating toolentered the second state from the first state, the operation estimationunit notifies to the event processing unit an instruction for performingthe feedback process for the operation input.

(3)

The information processing apparatus according to (2), wherein theoperation estimation unit estimates the operation input based on atleast one of the number and position of operating tools in the firststate.

(4)

The information processing apparatus according to (2) or (3), wherein,when a plurality of operation inputs are estimated by the operationestimation unit, the event processing unit determines a process to beperformed in the background based on an execution priority.

(5)

The information processing apparatus according to (4), wherein theexecution priority is determined based on a status of the operating toolin the first state.

(6)

The information processing apparatus according to any one of (1) to (5),wherein the event processing unit determines a target to be taken intoconsideration when performing a process in the background, based on astatus of the operating tool in the first state.

(7)

The information processing apparatus according to any one of (1) to (6),wherein the first state is a state in which the operating tool is inproximity of a detection surface, and the second state is a state inwhich the operating tool is in contact with the detection surface.

(8)

The information processing apparatus according to any one of (1) to (6),wherein the first state is a state in which the operating tool is incontact with a detection surface, and the second state is a state inwhich the operating tool is pressing the detection surface with a forcehigher than predetermined pressure.

The present application contains subject matter related to thatdisclosed in Japanese Priority Patent Application JP 2011-118573 filedin the Japan Patent Office on May 27, 2011, the entire content of whichis hereby incorporated by reference.

1. An information processing apparatus comprising: an event processingunit for performing in a background a process corresponding to anoperation input that is estimated based on a state of an operating toolin a first state, and for performing, when the operating tool in thefirst state enters a second state, a feedback process for the operationinput using the process performed in the background.
 2. The informationprocessing apparatus according to claim 1, comprising: a detection unitfor detecting a state of the operating tool; an operation estimationunit for estimating, based on a detection result of the detection unit,an operation input by the operating tool; and an operation statusdetermination unit for determining, and notifying to the operationestimation unit, a change in a state of the operating tool that is inthe first state, wherein, when the operating tool is determined based onthe detection result of the detection unit to be in the first state, theoperation estimation unit estimates, and notifies to the eventprocessing unit, an operation input by the operating tool, and wherein,when the operation status determination unit determines based on thedetection result of the detection unit that the operating tool enteredthe second state from the first state, the operation estimation unitnotifies to the event processing unit an instruction for performing thefeedback process for the operation input.
 3. The information processingapparatus according to claim 2, wherein the operation estimation unitestimates the operation input based on at least one of the number andposition of operating tools in the first state.
 4. The informationprocessing apparatus according to claim 2, wherein, when a plurality ofoperation inputs are estimated by the operation estimation unit, theevent processing unit determines a process to be performed in thebackground based on an execution priority.
 5. The information processingapparatus according to claim 4, wherein the execution priority isdetermined based on a status of the operating tool in the first state.6. The information processing apparatus according to claim 1, whereinthe event processing unit determines a target to be taken intoconsideration when performing a process in the background, based on astatus of the operating tool in the first state.
 7. The informationprocessing apparatus according to claim 1, wherein the first state is astate in which the operating tool is in proximity of a detectionsurface, and the second state is a state in which the operating tool isin contact with the detection surface.
 8. The information processingapparatus according to claim 1, wherein the first state is a state inwhich the operating tool is in contact with a detection surface, and thesecond state is a state in which the operating tool is pressing thedetection surface with a force higher than predetermined pressure.
 9. Aninformation processing method comprising: performing in a background aprocess corresponding to an operation input that is estimated based on astate of an operating tool in a first state; and performing, when theoperating tool in the first state enters a second state, a feedbackprocess for the operation input using the process performed in thebackground.
 10. A computer program for causing a computer to function asan information processing apparatus including: an event processing unitfor performing in a background a process corresponding to an operationinput that is estimated based on a state of an operating tool in a firststate, and for performing, when the operating tool in the first stateenters a second state, a feedback process for the operation input usingthe process performed in the background.